Prababilistic Instance-Based Learning

Traditional instance-based learning methods base their predictions directly on (training) data that has been stored in the memory. The predictions are based on weighting the contributions of the individual stored instances by a distance function implementing a domain-dependent similarity metrics. This basic approach suuers from three drawbacks: com-putationally expensive prediction when the database grows large, overrtting in the presence of noisy data, and sensitivity to the selection of a proper distance function. We address all these issues by giving a probabil-istic interpretation to instance-based learning , where the goal is to approximate pre-dictive distributions of the attributes of interest. In this probabilistic view the instances are not individual data items but probability distributions, and we perform Bayesian inference with a mixture of such prototype distributions. We demonstrate the feasibility of the method empirically for a wide variety of public domain classiication data sets.